ECE 6605 - Information Theory

Prof. Matthieu Bloch

Tuesday, December 5, 2023

Announcements

Project
- Send me an email to confirm your choices!
- I'll hold project office hours to help you
- This is meant to be fun
Exam options
- Oral: list of problems published, scheduling options coming
- Written: Thursday, December 14 11:20 am - 2:10 pm
Last time
- Secret key generation
- Covert communications
Today
- Information theory beyond this class

Lossy Source coding with side information

What if we only have limited rate for communication?
- Lossy source coding: we can't reconstruct exactly
- Need to accept some distortion: $d : X \times U \to [0, d_{max}]$ , $d (X^{n}, U^{n}) = \frac{1}{n} \sum_{i = 1}^{n} d (X_{i}, U_{i})$
- Require $lim_{n \to \infty} E_{} [d (^{n}, U^{n})] \leq D$

The rate–distortion function for $X$ with side information $Y$ available at the decoder is $R (D) = min (I (X; U) - I (Y; U))$ where $U - X - Y$ and the minimum is over all $p_{U | X}$ and functions $\hat{x} (u, y)$ such that $E_{} [d (X, \hat{X})] \leq D$ and $D_{min} = min_{\hat{x} (y)} E_{} [d (X, \hat{x} (Y))]$